4b code base

• [X] Can you spawn a worker to prepopulate the history in pourover with some examples so I make the plots by hitting "up". eg: already "plot("selJets.pt")" appears grayed out on start up. It would be good if I can access that command in the history, can you also add other options where I rebin or use yscale="log" and do region = "sum"
• spawn a worler for …
• [X] Veriffy … Looks good!

• Fixing the iPlot and pourOver locally

• Ran a local test of new SvB histogrmas (and probably also new Jet Hists)
• [>>] Check hists

• cleaning up the unneeded HF candidates
• adding histograms for the extra cand vars
• [X] Want to make list of SvBs configurable…
• Now the SvB hists are discoverable based on the friend list
• [>>] Verrify the new plots are being made

• Doing the ak2 refactor … Done… Big speed up in the thrust calculations apparently

• Migrating akward->akward2 … in cworker
• Making a new CI for Run3 mixing coverage … added … committed
• [X] Planning a refactor from on ak2
• Adding a full Run3 CI with the cutflow check before ak2 refactor.. merged

• Thinking of a better way to compute the normalization parameter in the JCM fits… now floating it…. Leads to much better fits

• FeynNet fixed !!!!

• Re running histogrmas…
• No change..
• Debuging weights… found that the SvB_FeynNet scores are unchanged
• Looks like lpc had stale code when the weights were remake…
• Remaking data friends on cmslpc335… done
• Remaking hists on falcon

./run_container snakemake --snakefile coffea4bees/workflows/Snakefile_Run3.smk --profile software/snakemake/profiles/falcon --config mode=quadjet_run2 --jobs 20 --logger snkmt --rerun-triggers mtime  --allowed-rules make_histograms merge_histograms make_histograms_wJCM merge_histograms_wJCM make_plots_wJCM make_histograms_FvT merge_histograms_FvT make_plots_FvT create_friends_wSvB create_friends_FvT create_histogram_config_wSvB create_histogram_config_wJCM create_histogram_config_FvT all_histograms all_FvT_histograms -- all_histograms all_FvT_histograms

• Think I found the issue in the FeynNet discrepancies… we are not vetoing fwd jets that are higgs cand jets.
• Rerunning SvB_FeynNet friends
• data only
• Resubmitted data on cmslpc354

./run_container snakemake --profile software/snakemake/profiles/lpc --snakefile coffea4bees/workflows/Snakefile_SvBFeynNet_friendtrees_Run3.smk --cores 4 --logger snkmt --retries 2 --rerun-triggers mtime --forcerun make_SvBFeynNet_friendtrees_data 

• Adding HH to the SvB friendtree processing
• New Feynet distributions look reasonable
• Remake FvT and MvD studies with new

• Update fourTag selection in Run3 to 3T1M
• Will add a configuration flag which we will turn off when doing the synthetic or mixed data
• [X] Dont want to affect mixed hemisphere making
• [X] Synthetic data making
• Added Tight selection and 3T1M fourTag definition
• Merged
• Good idea re: parking and pre-parking…
  • for data will set a flag based on the run number if >XXX is parking
  • for MC will create friend tree to randomly assign based on lumi weights
• Need:
  • [X] lumis per era preparking = 5.613751282 / parking = 12.350466716 fraction = 0.687505683467
  • [X] Max run number
• Code wired up
• CI passed
• Runnign Snake make to make the classifier is_parking MC inputs… on tmux… cmslpc343

./run_container snakemake --profile software/snakemake/profiles/lpc --snakefile coffea4bees/workflows/Snakefile_isParking_friendtrees_Run3.smk --cores 4 --logger snkmt  

• failed… fixing … rerunning… finished

• Remaking SvB_Feynet Friends

  ./run_container snakemake --profile software/snakemake/profiles/lpc --snakefile coffea4bees/workflows/Snakefile_SvBFeynNet_friendtrees_Run3.smk --cores 4 --logger snkmt 
  

• Finished…
• Now need to test with 4M and with 3T1M selection…

• Thinking about 3T1M…
• Right now mixing done for 4M … but anaylysis wants 3T1M … will come back to this after solid Run3 CI tests
• Working on Run3 CI tests

• Plotting differnt vars from different files… Done

• Starting slurm_monitor.py
• Need to test … working

• Looking into 2d plots
• OK 2D plots now working …
• Full works
• ratio works

SnakeMake monitor:

• should we make this code a fork ? … no we import instead
• Testing with my rerun
• adding features

• Trying to get a good monitor setup
• settting up src/tools/runner_monitor.py
• Got the counts
• Got it running !
• Now need to integrate it in the TUI
• devleop barista_console.py

~/.pixi/bin/pixi run python src/tools/barista_console.py

• Progress bar showing up !
• Can run on other machines !

• Error message for For condor_monitor

    condor_tail -name lpcschedd5.fnal.gov 58603883.0
  Failed to peek at file from starter: Invalid symlink or filename (/dev/null) outside sandbox (/storage/local/data1/condor/execute/dir_2947429)
  

• Getting the container to run locally

• Run with

  limactl shell cms
  cd CMSWork/hh4b/barista
  ./run_container ...
  

• Long time to install…
• Got it running but it takes a long time to load… not sure its worth it

• Start condor_monitor based on https://github.com/patrickbryant/nTupleAnalysis/blob/master/python/condor_monitor.py

• More cleaning up plotting… Big restructuring

• Removed passPreSel from cut lists
• (All in claude)
• Refactoring plotting code .. big changes!

• Chatting to AE re:killing years
• removing passPreSel … factor of 2 for no reason
• [X] Updating the plotting code to not require a cut…
• remove passPreSel as default
• added option to fail cut with ~cutName
• [>>] Move defaults to not use histCut

• [X] Idea about saving memory by using one year
• Maybe we can also remove cutHists

• Cleaning up plotting
• Created /test-barista and /test-coffea4bees skills… works great!

• Can switch accounts with /login
• [X] Add SnakeMake workflows for all the CIs "Our gitlab CI workflows are in coffea4bees/workflows/gitlab-CI/ most of these also have Snakemake workflows in Snakefile_testCI.smk. Can you find and add any that are missing ?"
• [X] Cleaning up the handing of firends Clean up Run2 vs Run3 configs " I want to clean up the analysis/metadata config files. Specifially there are some configs with a nominal version and a Run3 version where the only real difference in the friends configuration eg: HH4b_{classifierinputsRun3} and HH4b_{classifierinputs}. I would like to create a seperate config file for friends that changes with the year, similar to what we do in coffea4bees/metadata/triggers_HH4b.yml. We then need to make sure that file is getting parsed and the friends are properly passed in runner.py
• Claude Coded… Testing…

• Setting up config of object selection
• Migrate object selection to yamls
• Moving the processors to use the base class:
  • I did processors_cluster4b
  • Claude does make_hemilibrary
  • Then claude cleans up cluster_4b…
• [X] split out cand_jet
• [X] clean skimmer codes
• [X] Fix Hard coded quadJet selection
• [X] Clean up process_studyhemis

• ./run_container source coffea4bees/scripts/analysis-test.sh
• ./run_container source coffea4bees/scripts/analysis-signals-test.sh
• ./run_container source coffea4bees/scripts/analysis-merge-test-job.sh
• ./run_container source coffea4bees/scripts/analysis-plot.sh
• Debugging

• Setup code new_master
• Running CIs
• Making plots… But I dont think the tag is being used
• Now fixed tag

• Testing 2d plots
• Fixing CI
• Merging with master
• clean up interface
• Split making the dict and plotting

• Working 2d plots
• Merged the 2d and 1d helper functions
• Got the ratio going !!!

• Roc Plots
• Bug fix phh
• rerunning signal samples

• Working Roc Plots
• Run to make phh plots (SR and HH region)
• Added kl reweighting function
• Spikes from one of the variances of the signal sample… need to request nominal
• GOT kl-rw working!!!
• Added new hists (phh and phh_hh)
• Running all signals on cmslpc322
• lots of errors… fixing
• Running on condor
  • =source scripts/analysis-testAll-job.sh =
  • some ZZ4b file transfer failed
  • resubmit
• Signal finished
• Finally finished backgrounds too
• p(HH) seems even better than p(S) !!!
• mailed AE
• push code
• Add to AN
  • <font color=red> On Saturday, on Saturday … on Saturday </font>

• Working Roc Plots
• got one curve at a time.
• Got multiple curves going
• Need to plot from different files
• python plots/makeRocPlot.py hists/histAll_20250304_a100c66.coffea hists/histAll_signals_cHHHX_20250304_a100c66.coffea --outputFolder roc_curves
• [>>] Run to make phh plots (SR and HH region)
• Curves for different k-lambdas
• Found spikes in kl = 2.45 sample